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ABSTRACT 


This paper statistically analyzes 15 years of NAVAIRSYSCOMHQ 
fixed-price incentive contract experience in the aircraft and missile 
procurement field. The relation of basic contract parameters to con- 
tract outcome is explored through regression and analysis of variance 
techniques. 

The inferences arising from the statistical analysis are combined 
with other information to draw conclusions regarding incentive contract- 
ing. The most important of these is that there is no evidence that the 
negotiated sharing ratio has any influence on the contractor during the 


merrormance of tne contract. 
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I. INTRODUCTION 


A. FIXED-PRICE INCENTIVE CONTRACTS - THEORY AND PRACTICE 
Plc ony (from the governnent's viewpoint ) 

The basic parameters of a fixed-price incentive (FPI) contract 
are: target cost, range of incentive effectiveness (RIE), sharing 
ratio, ceiling price, and target profit. 

Fixed-price incentive contracts are intended to be used for 
engineering development, management support, and production contracts 
in which the uncertainty of cost is too large to allow a firm fixed- 
price (FFP) contract. In terms of cost, this means approximately ap 
15 to 20% variation about the negotiated target cost. If the range 
of cost uncertainty is larger than this, a cost reimbursable contract 
18 indicated. If significantly smaller, a FFP contract should be 
considered. 

FPI contracts are further subdivided into those involving a 
cost incentive alone, and those having schedule and/or performance 
incentives as well as a cost incentive. Both the cost incentive and 
multiple incentive contracts may have firm target(s), FPIF, or suc- 
cessive targets, FPIS. The contracts analyzed in this thesis are FPIF 
and FPIS contracts with cost incentives only. 

Theory has it that the contractor will be influenced to effrec- 
tively control costs and to make cost-associated trade-off decisions 
in a way favorable to his profit and therefore favorable to the govern- 
ment. The government's desires with regard to the cost outcome of a 
contract are communicated to tne contractcr in the form of the mutually 


agreed upon sharing ratio. This ratio, e.g., 80/20, says to the 


6 





contractor that his profit will increase $.20 for every dollar he saves 
in cost throughout the RIE, and vice versa. The RIE is chosen so as to 
encompass all likely contract cost outcomes including the most likely 
outcome (the target cost). Insurance against a massive cost overrun is 
provided by another parameter, ceiling price. This figure tells the 
contractor the maximum the government will pay, cost + profit, for com- 
pletion of the contract no matter what the cost outcome. It should be 
noted that for a cost outcome greater than the ceiling price, the con- 
meactor s prolib 1S negative, 1.e€., a loss. Also for a range of cost 
outcomes less than the ceiling price, each dollar of extra cost comes 
out of the contractor's profit directly. In effect, the sharing ratio 
in this range is 0/100. This range is bounded by the point of total 
assumption (PTA) on the lower cost side, and unbounded on the higher 
Cost Side. 

The target profit is established upon, at least in part, the 
degree of financial risk accepted by the contractor as eee dace 
the type of contract and where applicable, the sharing ratio. That: , 
an FPI contract with a 50/50 sharing ratio is assumed to be more fine - 
cially risky for the contractor than one containing an 80/20 sharing 
ratio. Further, a FFP contract is considered more financially risky 
than a cost plus incentive fee (CPIF) contract. 

The underlying assumptions of incentive contracting are: 

a. That the contractor seeks to maximize profit on each 
Concrace. 

Dawe inat the most likely, optimistic, and pessimistic cost 
outcomes are established free of bias and with some degree of accuracy. 

c. That the contractor can control the cost of performance 


Significantly during performance. 








Zee Prac wiles 
In practice there is reason to believe that each of the previous 
assumptions is subject to considerable doubt. 

a. Profit maximization - Contractors appear willing to 
sacrifice profit on a particular contract to other goals such as long- 
term profit or simply survival. 

b. The most likely cost outcome, target cost, is subject 
to a number of types of bias. For example, a contractor may be willing 
to accept an unrealistically low target cost if the industry is charac- 
terized by overcapacity and high competition. Similarly, the government 
may accept an excessively optimistic view of future cost outcome if the 
competition for available funds is high within the government. The 
target cost negotiated in a sole-source situation may be unrealistically 
high if the government has no good basis of comparison. In which case 
the contractor can increase his profit by “accepting” an FFP contract 
and gain a 0/100 share of the underrun and a higher target profit to 
boot / ref. ie And lastly, it is not currently possible to predict 
future costs with any reasonable degree of accuracy on a technically 
complex item which has not been produced before. Further, contract 
negotiation is an adversary proceeding in which each side does its best 
to protect its own interests. Such a proceeding cannot be characterized 
as a cooperative search for truth in the establishment of contract tar- 
get cost. For example, the contractor may raise his negotiation objec- 
tive with regard to target cost to compensate for the risk of loss 
imposed by the existence and value of the sharing ratio. The selection 
of three costs, pessimistic, most likely and optimistic, does not de- 
Bede the uncertainty in the cost estimate unless valid information 


1s available upon which to base the cost probability distribution 








inherent in the three costs. In the author's opinion, such information 
is generally not available and the seeming decrease in cost uncertainty 
a5 1)lusory . 

c. Contractor control of cost during performance - 
In order to control costs during performance, the contractor must be 
able to predict cost outcomes sufficiently in advance and take effec- 
tive action. Attempts by a contractor to make significant improvements 
in efficiency during performance of a contract may result in managerial 


disutilities and may increase rather than decrease costs / ref. Die Ge 


B. OBJECTIVE OF THE ANALYSIS 

The objective of the analysis presented subsequently was to draw 
statistical inferences as to the readily quantifiable contract para- 
meters which have a statistically significant effect on contract out- 
come in percent deviation from target cost. The chosen parameters are 
ones that are measurable and known at the time the contract is signed 
or shortly thereafter. These parameters are target fee, sharing ratio, 
ceiling price, target cost, scheduled length of performance, number of 
articles procured, the number of aircraft and missile contracts initiated 
that year, and the year performance started. As originally scheduled 
contract Be was not available, actual length of performance 
was substituted therefor. 

A further objective of this analysis was to generate, if possible, 
a model which explained a significant amount of the variation from 
contract to contract in the outcome percent deviation from target cost 
using the contract parameters listed below. The utility of such a 


model is obvious in the predictive sense. 





Percent deviation from target cost was determined to be the depen- 
dent variable for most of this analysis because of the three measurements 
of contract outcome, cost, schedule, and performance, cost is the one 
which is most readily measured and publicized. Further, the adverse 
effects of system performance degradation from initial estimates and 
sehedule slides are difficult to identify and measure. 

The following factors in contract outcome may have been important 
but were unavailable: 

1. Initial contractor estimates of contract cost. 

2. Initial governmental estimates of what the contract should 
cost. 

: 3. The degree of technical risk involved in the article 
purchased. 

4. The changes in contract cost generated by schedule changes, 
engineering change orders, governmental failures to deliver government 
furnished data, mterial and equipment on time. 

S. Renegotiation of fees. 

These unmeasured variables and others can be expected to result in 
a rather large random variation in contract outcome as regards devia- 
tion from target cost. If this esta is indeed random rather than 
systematic and is distributed approximately normally, useful inferences 
can be drawn from regressions and etc., that do not explain a large 


part of the variation. 


C. HYPOTHESES 
In the opinion of the author, the following hypotheses could reason- 
ably be made regarding the effects of the measured contract parameters 


listed in B. on percent deviation from target cost: 
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1. Length of performance of the contract - it might reasonably 
be expected that cost deviation would increase (that is positively or 
in the direction of cost overrun) as the length of performance increases. 
This might be true because of the difficulty of projecting costs into 
the far future, the general inflationary trend experienced, and the 
increased opportunity afforded to make changes.. 

2. Target profit ~ Expectation would be that larger target 
profits would be associated with larger positive deviations from target 
cost because theoretically target profits are established on the basis 
of the risk involved in ee contract and other factors. 

3. Sharing ratio - Actual costs moved in the negative direction 
(toward cost underrun). The theory of cost incentive contracting is 
that the stimulus provided by an increased sharing ratio will cause the 
contractor to at least attempt to operate more efficiently. Of course, 
it may also stimulate him to negotiate for a higher target profit and/or 
a higher target cost. 

lh. Ceiling price - A higher ceiling pitice would be expected 92 
result in less attention to incurred cost and thereby tend to influen > 
deviation from target.cost in a positive direction. All other things 
being equal, a higher ceiling price (in terms of percent of target 
cost) moves the point of total assumption further from the target cost. 
If the actual cost of performance exceeds the point of total assumption, 
each dollar of additional cost comes directly from the contractor's 
profit on the contract. In effect the contract becomes a firm fixed-~ 
price contract (100% sharing ratio) beyond the point of total assumption. 

5. Target cost - The author does not care to hypothesize on 
this variable's effect on deviation from target cost. As the dependent 


variable in this analysis is the ratio of cost deviation to target cost 
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Ss 


expressed in percent, all other things being equal, increases in target 
cost would result in a decrease in the dependent variable. On the 
other hand, to the degree that an increased target cost may reflect 
miler eascd complexity, ancreased time of performance, etc., an increase 
in target cost might result in a positive change in deviation from 
target cost. Notwithstanding the clouding effect that choosing such 

a ratio for the dependent variable has on this independent variable, 
target cost, it was considered necessary to permit meaningful compari- 
son among contracts which vary in target cost from under $1 million to 
over $200 million. 

6. Number of articles procured - Deviation from target cost 
was expected to vary negatively as the number of articles procured, 
i.e., the larger the number of articles procured the less positive or 
more negative cost deviation was expected to be. The larger the number 
of articles procured under a particular contract the less the develop- 
mental aspects of a basically production contract should effect actual 
total cost. Further, larger production quantities would be expected 
to result in operation on more favorable portions of the learning curve 
for more of the contract performance. 

¢. Number of Navy aircraft and missile contracts signed the 
same year as a particular contract - It was expected that the larger 
the number of contracts signed, the less positive or more negative de- 
viation from target cost would be. The diversion of facilities and 
personnel from defense to non-defense work is at best a painrul and 
expensive alternative for major aerospace contractors. To a lesser 
degree the diversion from aircraft and missile production to other 
defense production is alsc painful and expensive. In Some cases, e.¢@., 


Grumman, the diversion from Navy aircraft and missile work to similar 
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work for the Air Force or Army is not particularly attractive. These 
things are considered true irrespective of the size of the contractor's 
investment in aircraft and missile production facilities because of 
the psychological barrier to entering a field where the rules of engage- 
ment are largely unknown from a field in which one is expert. Further, 
a Rand Corporation report indicates that at the same time that the num- 
bers of Navy aircraft and missile contracts were decreasing a like 
phenomena obtained in Air Force contracts / ref. Bay aes least as regards 
total contract dollar value. In sum then, exit from the Navy aircraft 
and missile market and entry into another was not easy so the effect of 
E decreasing number of available contracts should be a tendency toward 
a more optimistic view on the contractor's part of estimated costs 
particularly in competitively negotiated procurements. If such were the 
case, the final result could be expected to be a positive change in de- 
viation from target cost. | 

8. The year the contract was signed - The author's intuitive 
expectation was that contracts signed in the later years of the period 
covered by the data would show more positive deviation from target cost. 
ane steadily increasing publicity concerning cost overruns during the 
period would indicate that that was the case. The other variables! 
independency of the year the contract was signed is certainly open to 
question. Any of the other independent variables may prove to be time 
dependent which will reduce this variable's usefulness as a predictor. 
Other than as a reflection of changing contractor and/or government 
policy, the author is hard pressed to explain a systematic variation 


of deviation from target cost with time if such, in fact, exists. 
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Ii. DESCRIPTION OF DATA 


A. SOURCE 
The raw data were obtained by the author through the good offices 


of Mr. M. BE. Biciocchi, Naval Material Command, code O22B. 


B. NATURE OF THE DATA 
oe lori 
The data encompass all Naval aircraft and missile fixed-price 
incentive contracts completed, redetermined and approved by ONM during 
the period 195) through 1966. The information listed in the data for 
Edemecomuract 15% 
a. Contract period 
b. Contract number 
ec. Contractor 
d. Nomenclature and number of items procured 
e. Negotiated target cost 
f. Final cost 
ge Percent deviation from target cost 
h. Incentive profit pattern in terms of: 
(1) Target profit 
(2) Sharing ratio 
(3) Ceiling price in percent of target cost 
i. Final profit rate 
je Target profit 


ieee final prorit 


1h 





€. Quantity of Data 


The data contained 178 individual data points (contracts) from 
which field one was deleted because it dealt with the procurement of 
Spare parts. Thus the population analyzed contained 177 contracts. 

3, weer rod 

Although the earliest contract in the data was signed in 199 
and the latest in 1963, it was considered that results useful in the 
future could be obtained therefrom. It would appear that future Navy 
procurement will more closely resemble that of the 199 through 1963 
time frame than the later McNamara era. ae is the author's opinion that 
data taken from Mr. potenenae conan package procurement contracts would 
be of little value in attempting to draw inferences useful in the future 


when total package procurement is so vigorously eschewed. 


CGC. DATA SUMMARY 


1. Variable Means ard Standard Deviations 


Standard 
Variable Mean Deviation 
Contract period (months) 60.5 19.3 
Target profit (percent) 9.0 0.8 
Sharing ratio (percent) 19.3 8.1 
Ceiling price (percent) | teenie 5.1 
Target cost (millions of dollars) h6.5 16.6 
Number of articles procured 377 128 
Number of contracts signed per year i 8 
Deviation from target cost (percent) -1.3 9.1 


2. Comment on Variables 
Although target profit is intended to vary with sharing ratio 


and other factors, in fact, approximately 68% of negotiated target 
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profits fell within less than one percent of the mean, 9%. This is 
surprising considering that the data covers a period of 15 years and a 
multitude of different procurements. It is tempting to suspect that 
target profit was close to a "magic number" with a mode of, say, 10%. 
Ceiling price seems to fall rather neatly into the same category 
since the standard deviation is only five percent, and the mean, 123.2%, 
falls close to the mode, 125%. 
3. Miscellaneous 
The data includes purchases of combat and utility piloted air- 
planes, helicopters, blimps, drones, missiles, and airborne electronic 
equipment, e.g., radars. Twenty-three contractors are represented in 
the data. Of these, thirteen completed five or more contracts during 
the period. Six completed 10 or more contracts: 
Ling-Temco-Vought (10 contracts) 
Convair (10 contracts) 
North American (13 contracts) 
Douglas (23 contracts) 
Grumman (2) contracts) 
Lockheed (30 contracts ) 
On the contracts contained in the data, the Navy spent a total 
of $ 8,885,589,000 of which $ 8,13,30,000 were actual costs and 
$ 751,249,000 actual profits. The average profit rate realized by the 
contractors was 9.2% of actual costs. During the period of the data, 
defense contracting, at least in this area, was an extremely profitable 
business even on the basis of sales. Return on investment figures 
would, of course, be even more attractive given the large amount of 


government owned facilities involved. 
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ITt. DATA ANALYSIS METHODS 


A. LINEAR REGRESSION 
eeeelneo try. 
Jinear regression is a technique for making statements as to 

the degree of linear association between a criterion (dependent) and 
one or more predictor (independent) variables using the method of least 
squares. If evidence exists that suggests that the form of one or more 
independent variables should be of a higher degree, the original vari- 
ables can be transgenerated to fit the linear assumption. The following 
assumptions are required in order to make probability statements about 
Single or multiple linear regression: 

a. That the relation between the expected value of the 
dependent variable ( Y ) and an independent variable (X) (or X° or x! .3 
ONG XK, °X, or X4/Xo5 etc., as previously mentioned) is linear. 

b. That the error distribution is constant or homosceda: ic. 

c. That no variable which is apt to effect the dependent 
variable is excluded from the equation. This assumption is not impo: - 
tant if the objective is forecasting. 

d. That the differences between actual and estimated values 
of the dependent variable are normally distributed. 
Though these assumptions are not always justified in this analysis, the 
insight to be gained from the linear regression is considered more im- 
portant than precision. 

2. Computer Code 
The computer code used by the author for single and multiple 


linear regressions on the dependent variable was a part of the 
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International Business Machines Scientific Subroutine Package labelled 
REGRE. REGRE provides the following output for a particular regression: 

a. A list of the dependent and independent variables inclwied 
in the regression along with their means and standard deviations. 

b. A simple correlation coefficient (r) for cn independent 
variable which measures the degree to which variation in the dependent 
variable is linearly associated with variation in that independent 
variable. The simple correlation coefficient has a range from -1 to +1. 
Larger absolute values of the simple correlation coefficient indicate 
greater correlation while the sign indicates dependent variable varia- 
tion in the same direction as the independent variable (+) or opposite 
to the independent variable (-). 

c. The regression coefficient determined for each independent 


wee, 


variable included in the regression (b, 5d 

d. The standard error for each regression coefficient. 

e. A measure of assurance that each regression coefficient is 
not statistically different from zero by use of the Student t statist c. 

f. The value of the estimated dependent variable when the ir- 
dependent variables all equal zero (intercept). 

g. The multiple correlation coefficient (R) from which the 
coefficient of multiple determination (R°) may be obtained. Ro ranges 
between O ard 1 and represents the proportion of variation in the de- 
pendent variable that is accounted for by the net linear association 
of all of the independent variables included in the regression. 

h. The standard error of estimate for the regression. 


i. The degrees of freedom, sums of Squares and mean squares 


associated with an analysis of variance for the regression and the 





resulting Snedecor F statistic which infers the degree of assurance one 
may place in the fact that there is a systematic linear variation of 
the dependent variable with the included independent variables. 
3- Rationale for Linear Regression 

Although there is some support in the literature for a non- 
linear association of sharing ratio with deviation from target cost, 
the analysis is based on theories of risk aversion or easy exit from 
the defense market / ref. h / which the author believes to be unrealis- 
tic. The author's opinion on the ease of exit from or entry into the 
defense market has already been stated in section I.C.7. In the 
author's opinion contractors are willing to accept a high risk of de- 
creased or no profits to buy into a contract because follow-on contracts 
can be expected to be profitable. Contractors may also expect to im- 
prove their profit position through change orders (constructive or 
otherwise) during performance of the contract. A toughening government 
position on this sort of activity is a creature of the late 1960's and 
oO Ss 

In short then, the author has chosen to confine himself, except 
in one case, to linear combinations of first-degree variables. 

4. Stepwise Linear Regression 

The form of multiple linear regression actually used by the 
author in the final analysis was stepwise regression. In stepwise 
repression, a sequence of miltiple linear regressions are computed in 
a stepwise manner. At each step one independent variable is added to 
the regression egqu2tion on the basis of which of the remaining (unused ) 
independent variables makes the greatest reduction in the error sum of 
squares. Equivalently, the variable added next is the variable which 


has the highest partial correlation with the dependent variable partialed 
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on the variables which have already been added; and equivalently it is 
the variable which, if it were added, would have the highest F value. 
The computer code used for stepwise regression was BMDO2R 
developed by the Health Sciences Computing Facility at the University 
of California at Los Angeles. BMDO2R also produces a correlation 
matrix of each independent variable with all of the other independent 
variables considered for the regression equation. It also outputs a 
plot of residuals versus computed values of the dependent variable for 
examination for systematic trends, if any. The intercepts, Multiple R, 
standard error of estimate, and analysis of variance are produced after 
E-c@ectep as in the linear regressiom™¢code discussed previously, REGRE. 
5. linear Regressions on Segments of the Data 
linear regressionswere performed for each contractor having 
five or more contracts during the period relating the year the contract 
was Signed and the number of contracts signed that year to predict values 
of percent deviation from target cost. 
The plot of percent deviation from target cost versus the year 
of the contract in the case of Lockheed revealed a sine wave pattern. 
A special linear regression was done on the Lockheed data using the 
cosine of a function of the year signed as the independent variable. 
6. Target Profit as a Dependent Variable 
To investigate the degree to which target profit depends upon 
the negotiated sharing ratio and length of contract, linear regressions 
were performed using target profit as the dependent variable and sharing 
ratio and length of contract respectively as independent variables. 
¢. Regression Independent Variables 
The following variables available in the data were considered 


to be suitable for linear regression: 
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B. 
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ANALYSIS OF VARIANCE 


1. Theory 


Description 


Length of contract period in months 

Actual cost deviation from target cost 

in percent of target cost (dependent 
variable ) 

Target profit in percent of target cost 
Sharing ratio in terms of contractor's 
share of the cost risk in percent 

Ceiling price in percent of target cost 
Target cost in millions of dollars 

The number of articles procured 

The number of Navy aircraft and missile 
contracts signed the same year as a 
particular contract 

The year the contract was signed, coded 

by subtracting 198 

The ratio of the target cost and the n ber 
of articles procured in millions of do lars/ 
article 

The product of the target cost and the 
number of articles procured in article- 


millions of dollars 


Analysis of variance is a way of dividing total variation in 


experimental data into components that can be assigned to specific 


Sources. The objective is to test the statistical Significance or 
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differences among average responses caused by some variable after making 
allowances for other variables. The label, analysis of variance, is 
appropriate because if the mean responses of the test objects are 
different among groups, then the variance among groups will exceed the 
independently computed within-group variance. 

Analysis of variance has some rather severe drawbacks when ap- 
plied to nonexperimental or observational data. First, when some 
classifications of the data contain one or less data point, a within- 
group variance can not be computed and the mathematical rigor of the 
analysis is severely reduced. This problem can be ameliorated by 
pooling of some treatments such that several data points fall in each 
classification. Second, the basic analysis does not provide informa- 
tion as to which treatment(s) were statistically different. ‘The F test 
performed only indicates that one or some of the populations were 
actually different. Further investigation as to which treatment or 
treatments were actually different from the others can be conducted 
using the Scheffe comparison test which will be discussed later. Thi i, 
gradual changes in the means in the data through the treatments will 3 
discovered in the analysis of variance but not be isolatable by the ¢< .b- 
sequent comparison test. In this case, the effect of the particular 
variable may be investigated through regression or a combination of 
regression and analysis of variance, analysis of covariance. 

The basic assumptions of the parametric analysis of variance 
are: 

a. That the observations are independently drawn from normally 
distributed populations. This assumption makes the analysis parametric. 


b. That the populations all have the same varience. 
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c. That the means in the normally distributed populations are 
linear combinations of "effects" due to rows and colums, i.e., that 
the effects on the dependent variable of the factors included are 
additive. 

2. Computer Code 

The computer code used by the author for one-way analysis of 
variance was BMDOIV developed by the Health Sciences Computing Facility 
at the University of California at Los Angeles. BMDOIV cutputs: 

a. Treatment means and standard deviations 

b. Within-groups, between-groups, and total sums of squares. 

c. Within-groups, between-groups, and total degrees of freedom. 

d. Within-groups and between-groups mean squares. 

e. F ratio for the test of the hypothesis Ho? mean, = means 
=" " "= mean, « 

More than one-way analyses of variance were not used by the 
author because of the prohibitive severity of the empty cell or Bal 
observation cell problem which arises in multi-way analyses. For ex: 
ample, a three-way ANOVA on contractor (there are 23 of them), year 
(15 of them) and item (7 of them) would contain 23 X 15 X 7 = 2h15 
cells. Since there are 177 data points, at the least, 215 - 177 = 
2238 would be empty, and the majority of the filled cells would con- 
tain only one data point. 

3. Rationale for Use 

The author's rationale for employing a parametric analysis of 

variance is essentially that stated under linear regression. 
h. Variables 


The factors (variables) chosen for one-way analysis of variance 


were: 
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a. Factor one - contractor with the individual contractors 
being the levels of that factor (contractors that had more than one 
contract during the period). 

b. Factor two - type of equipment procured using the following 
levels: 

OP epigllabint alte o Ienetaps 

(2) Combat airplanes 

eve tiissi les 

() Blimps 

(5) Helicopters 

(6) Pilotless airplanes (drones) 


(7) Airborne avionic equipment 


C. SCHEFFE COMPARISON TEST 


es Theory 


If an analysis of variance has resulted in the rejection of the 
equal mean hypothesis, the Scheffé comparison test (S-test) may be used 
to compare levels two at a time (or more) to identify which levels were 
Significantly different from others, if any. Again, a gradual change 
over the levels will not be identified. 

2. Computer Code 

Because a preprogrammed routine for the S-test was not found 
available by the author, a computer code was written by the author. 
That code is contained in Appendix A. No attempt was made to write a 
generally applicable code. The program contained in Appendix A is 
tailored to fit the data being examined in this thesis and to use the 


level means, etc., available from the preceeding analysis of variance. 
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Bee opp lt cati on 
Analyses of variance and Scheffe comparisons were conducted 
upon the entire data and some individual contractor data. Individual 
contractors so treated were those that completed enough contracts 
during the period to make such an analysis feasible. Specifically, 
the Lockheed, Grumman and Douglas data, classified by the year the 
contract was signed, and pooled as necessary to have at least two 


observations per group, were so analyzed. 
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IV. RESULTS OF DATA ANALYSIS 


A. LINEAR REGRESSIONS 
1. <All Contractors 
a. Stepwise Linear Regression 
The stepwise linear regression discussed in paragraph 
II.A.4. resulted in the following multiple linear regression equation: 
Percent deviation from target cost = -lj2.l + 0.126X, ~ 1.68, 
+ 0.397% 


eee ey (he, 0.823X, - 0.0000LX, , 


5 8 

The independent variables entered the equation in the order 
Xo, x, Xi; Xia; Xos Xo. When the stepwise regression routine termi- 
nated the computation, the lowest F-ratio for a variable included in 
the equation was 7.53li5. The highest F-ratio for a variable not in- 
cluded was 1.1953. When the routine was forced to start with each of 
the variables other than Ke the resulting equation was the same unless 
the routine was forced to include a variable which would not have been 
otherwise included because of low F-ratio. 


The linear regression variables are included here again to 


ease interpretation of the results: 


Symbol Description 
X, length of contract period in months 
X,, Realized deviation from target cost in 


percent of target cost (dependent variable ) 
X Target profit in percent of target cost 
xy Sharing ratio in terms of the contractor's 
share of the overrun or underrun 


Xe Ceiling price in percent of target cost 
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symbol . DeSscri pe lon 


Xe Tapeeurccss in meliiions of dollars 
Xo The number of articles procured 
Xp The number of Navy aircraft and missile 


contracts signed the same year as a 
particular contract 
Ko The year the contract was signed, coded 
by subtracting 19),8 
Kio The ratio of the target cost and the number 
of articles procured in millions of dollars/ 
article 
Xi, The product of the target cost and the 
number of articles procured in article~ 
millions of dolianes 
The F-ratio for the whole regression is 9.559 which indi- 
cates that the level of the test is higher than 0.005. That is, the 
probability that all of the regression coefficients are not zere is at 
least 0.995. 
The F-ratio test for each regression coefficient in the 


equation above along with the associated probabilities are listed 


below: 
Regression Probability 
Coefficient Value F-ratio b; #0 
b, 0.128 eo 0.995 
b, -1 .680 4.67 0.950 
Be 0.397 ome ee 
bp -.297 7.03 0.990 
by Ones 9.51 0.995 
b - .0000), (e568 0.990 


of 





The additional variables, X,. and Kia were formed from two 


10 
of the original variables, Xe and Xo, which were not significant in the 
regression individually and were not closely correlated with each other 


as determined from the correlation matrix. The correlation of Xe Wee 10 


xX was -.023. 


( 
ae X6/Xo 
The variables considered in the regression but not included 

were: 
Regression Probability 
Coefficient F-ratio bs #0 

b), 0.80 “less than 0.75 

be 0.68 i OS 

bo Cee uy i “O250 

Bio 0.90 ff TORS 


The value of the multiple correlation coefficient (R) fo. 
this regression was 0.502. Therefore, the value of the coefficient 
multiple determination (Ro) for this regression was 0.252, i.e., 
approximately 25% for the variation in the dependent variable (percent 
deviation from target cost) is accounted for by the net linear associa- 
tion of independent variables, Xi; X3, Xe, Xo, Xo and Xi4- 

The standard error of estimate for the regression is + 7.9/ 
percent deviation from target cost. That is, approximately 68% of the 
estimates from the regression equation fall within + 7.97 of the ob- 
served values of percent deviation from target cost whose mean was 


-1.25. 
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b. Percent Deviation from Target Cost Versus Year Contract 
Signed 
A linear regression was performed relating the percent de- 
viation from target cost to the contract year alone to get a historical 
perspective. The resulting equation was: 
% deviation from target cost = -l.9) + 0.573Xq 
The computed t-value for this regression was 3.28 or equivalently the 
F-ratio was 10.73, both meaning in this case that the probability that 
the value of the regression coefficient was actually zero was .005. 
The coefficient of correlation was .2l0 and the proportion of variation 
in the dependent variable accounted for is approximately 0.06. The 
standard error of estimate was + 8.82. 
Ga Target Profit (X,) Versus Other Contract Parameters 
A series of single linear regressions were conducted to 
determine if any other contract parameter known to both sides during 
negotiations was a good predictor of the negotiated target profit. 


The results of these regressions were as follows: 


Independent 2 Standard Probability 
Variable Intercept by R R° Error of Est. — bs #0 
X, 9.60 -.0102 eels 0.828 0.995 
X), On -0320 30h 09 0.811 0.995 
Xo 8.92  .00017 .25) .06 Oe23 0.995 
X¢ 9.07 -.00182 .100  .01 0.87 O71 5e 
Xp aot 30529 “guise 50 O200/ 0.995 


A multiple linear regression was performed using dependent 
variables X,, Kyi Xe and Xo. The results of this multiple linear 


regression were: 





Independent . Probability 


Variable b, t-value b, #0 

X, - .00),37 1.36 0.900 
X) .0182 230 0.975 
oy - .0395 Se) oe 5 
XK 0001 2.08 0.975 

Intercept 13437 

R 0.137 

Re 0.19 

Standard Error of Estimate 0.773 

F-value 8.06 

Probability that b, # 0 0.995 


Variable X¢ (Target Cost) was not included in the final 
regression equation because the t-value (.01),) did not have a high 
enough probability that the regression coefficient (be) was not equal 
to zero (less than 0.75). 

2. Individual Contractors 
a. Grumman 

The plot of percent deviation from target cost versus year 
of the contract for Grumman's contracts contained in Figure 1 displayed 
an apparent annual trend. This apparent trend suggested a linear re- 
gression on percent deviation using the year the contract was signed 


(XQ) as the independent variable. The results of this regression were: 


Intercept -12.3h 
Regression Coefficient (Do) 1.57 
Correlation Coefficient (R) 0.631 
Determination Coefficient (R°) 0.h0 
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Std. Error of Estimate +8.20 
F-value 14.5) 
Probability that bg # 0 0.995 
A similar linear regression of percent deviation versus the 
number of contracts signed the same year as the observed contract (Xe) 


was performed. The results were: 


Intercept 9.68 
Regression Coefficient (bg) -0.716 
Correlation Coefficient (R) 0.667 
Determination Coefficient (Ro) 0.5 
Std. Error of Estimate (08 
F-value 17.65 
Probability that bg # 0 0.995 


Because of the predictive power displayed by variables Xe 
and Ko in the case of the Grumman data, the author was moved to attempt 
to generate a Grumman model for percent deviation from target cost. 
This was done using the stepwise linear regression routine previously 
used for the same purpose with all of the data. 

One of the uncertainties of stepwise regression is that the 
resulting value of the multiple correlation coefficient (R) may vary 
depending upon the first independent variable used. This uncertainty 
becomes particularly important when the list of independent variables 
considered contains two rather powerful predictor variables (Xp and Kg) 
which are closely correlated (correlation of Xg with Xp = =Oe03G) in 
this case, the routine picks the most powerful variable and the other 


is more or less automatically eliminated from further consideration by 


its high correlation with the first variable used. To eliminate this 
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uncertainty of result, the stepwise regression was conducted with all 


independent variables free and then another stepwise regression was 


conducted in which the routine was forced to start with the previously 


unused variable. 


were: 


Regression 


Coefficient 


bg 


Bio 


Intercept 


The results of these two stepwise linear regressions 


(1) Ald Independent Variables Free 


Value 





-~0.688 
8.918 


Correlation Coefficient (R) 


Determination Coefficient (rR) 


Std. Error of Estimate 


F-ratio for the Regression 


Probability all b, 7 0 


Regression 


Coefficient 


Intercept 


(2) Routine Forced to Use X 


Value 





OPO 
=O © 


2.7h 


Correlation Coefficient (R) 


Determination Coefficient (R°) 


Std. Error of Estimate 


Probability 
F-ratio b; a0 
19.7) 0.995+ 
See 0.900+ 
4.95 
0.752 
0.57 
II, 
13.66 
0.995 
y 
Probability 
F-ratio b, #0 
Doe O. 750+ 
12286 0.995+ 
Bole 0.995++ 
(Ogee 
0.82 
0.66 
ome 
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F-ratio for the Regression W320 
Probability all bs #0 0.995++ 

b. Lockheed 

A plot of percent deviation from target cost versus the 

year the contract was signed for Lockheed (Figure 2) also revealed an 
interesting pattern. This pattern looked to the author like a sine 
wave. The independent variable for a linear regression was determined 
to be a function of the year the contract was signed (X = cos(360/12 
(Xp -~ 2) + 90)) where the period of the wave was approximated as 12 
years. The regression equation used was then % deviation = a + bx. 
The results of this regression and an ordinary linear regression for | 


comparison were: 


—— cos (360/12 (Xo = 2) + 90) Xo 





Intercept (a) -2.3 ~.33 

Regression Coefficient (b) Dn O22 

Correlation Coefficient (R) 387 Oa2e2 

Determination Coefficient (Ro) 0.15 05 

F-value 4.928 1.457 

Probability that b #0 0.950 0.75C 
Cane uvher 


Plots of percent deviation versus year for the other con- 
tractors displayed no discernable significant pattern. 

Linear regressions of percent deviation versus number of 
contracts signed that year for the other contractors failed to reach 


the probability that b # O required for significant statistics (0.950). 
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Be ANALYSES OF VARIANCE AND SCHEFFE COMPARISONS 
1. By Contractor 
a. Analysis of Variance 

Preparatory to conducting a one-way analysis of variance by 
contractor, the data field was reduced by removing the data for those 
contractors who had only one contract during the period. Each remaining 
contractor was designated as a treatment level of the factor, contrac- 
tor. The sample size , mean percent deviation and standard deviation 


for the contractors (treatments) analyzed were: 
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Sample Standard 

Treatment Size Mean Deviation 
Beech hy -2.78 1 2ecil 
Ling-Temco-Vought 10 0.59 9.38 
Convair 10 -.37 8.99 
Douglas 2h -2.11 9.25 
Goodyear 6 -),.70 lien > 
Grumman 2h =1.99 10.3) 
Hiller 8 fice Clo 
_ Kaman 6 11.95 12.97 
Martin 7 -6 .30 7-35 
McDonnell me 4.95 11.82 
North American 13 -1.75 6.07 
Vertol 3 eet 5.8 
Ryan 6 -2.33 3.60 
Sikorsky Z -6.82 4.60 
Bell 2 4.65 8.8), 
Lockheed 30 -2.50 4.53 





Sample otandard 


Treatment Size Mean Deviation 
Philco 2 ies Daler 
Raytheon 3 ~8.20 lea 7 


The F-ratio for this analysis of variance was 2.76 which 
allows rejection at a significance level of .005 of the hypothesis 
that the treatment samples all came from the same population. That is, 
one may be .995 confident that the treatment samples did not all come 
from the same population. However, the analysis of variance does not 
allow selection of any treatment as being significantly different from 
another. That is, one may not say that Philco with a mean percent de- 
viation from target cost of 1.25 is significantly different from 
Raytheon with a mean percent deviation of -8.20. 

b. Scheffe Comparison 

To continue the investigation started in a. above, a 
Scheffe Comparison test was conducted using the F-value corresponding 
to the level of significance designated by statisticians as statis- 
tically significant, .05. The result was that none of the treatments 
(contractors) were different from any of the others at that level. 
The level of the comparison was successively reduced until differences 
began to appear. At a significance of 0.25 there was still no statis- 
tical differences between contractors. At a significance level of 
0.50 differences began to emerge, but this is the statistical equivalent 
of a toss-up, i.e., these differences were no more likely to exist than 


i OmMOineek 1. SU « 
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2. By Type of Item Purchased 


a. Analysis of Variance 
To determine whether percent deviation from target cost 
were Significantly different among different types of items purchased, 
a one-way analysis of variance was conducted in which the type of item 
was the factor and the treatment levels were the seven different types 
of items purchased which have been previously identified. The mean 


percent deviations, sample sizes, and standard deviations obtained were: 


Samp le Standard 

Treatment size | Mean Deviation 
Utility Airplanes 21 —elh eee 
Combat Airplanes 88 -~0.98 8.52 
Missiles 19 =o Ni TOvee2 
Blimps 6 -),.70 by. 75 
Helicopters 28 2a, 11.08 
Drones 8 1.08 (eur 
Airborne Avionics 8 -1.18 9.50 


The F-ratio for this analysis was 1.5 which corresponds to 
a level of between 28 and .10. This level is insufficient to allow 
rejection of the hypothesis that the treatment samples came from the 
same ScuUAtiON: therefore further investigation using the Scheffe 
comparison test was not justified. 
3. Individual Contractors by Year the Contract Was Signed 

Three of the contractors were selected for separate analyses 
of variance on the basis that they had more than 20 contracts during 
the period. The year the contract was signed was selected as the 


factor for one-way analyses to determine if any sharp changes in 
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contractor policy or environment had occurred which would not have 
been identified when that factor was used as a linear regression 
variable. 
a. Lockheed 

(1) Analysis of Variance. Lockheed started and completed 
30 contracts during the period of the data. The Lockheed data were 
pooled into two-year treatments to provide adequate numbers of observa- 
tions in each cell and the odd contract was dropped. The sample sizes, 


means and standard deviations which resulted were: 


Sample Standard 
Treatment _Size_ Mean Deviation 
1949 & 1950 3 -0.73 De 
1951 & 1952 10 1.99 3.83 
ees 195) 5 =5.37 4.30 
1955 & 1956 hy 0.68 PINE: 
1957 & 1958 3 heey, Bed 
1959 & 1960 6 -3.07 3.38 


The F-ratio for this analysis was h.5) which allows 
rejection of the one~population hypothesis at a significance level oi 
.005. Because the hypothesis was rejected at a very significant 
level, the Scheffe comparison was performed. 

(2) Scheffe Comparison. A Scheffe comparison was con- 
ducted using the F-value corresponding to a significance level of «OS: 
"statistically significant". The pooled years 1951 and 1952 were de- 
termined to be significantly different from the pooled years 1957 and 
1958. However, in view of the sine-wave pattern exhibited by the Lock- 


heed data when plotted against the year the contract was signed 


By 





(paragraph IV.2.b), this appears to be a comparison of the peak of 
the wave with the valley. 
b. Grumman 
le Analysis of Variance. Grumman started and completed 
2h contracts during the pericd of the data. These data were pooled 
into three-year treatments and a one-way analysis of variance was 
conducted. The samples sizes, means and standard deviations which 


resulted were: 


Sample Standard 
Treatment _Size_ Mean Deviation 
oo = 1951 6 = 181515 (yee 
1952 - 195) 8 -1.59 8.443 
o> = 1957 I -0.70 NO 22 
1961 - 1963 hy 10.65 10.65 


The F~ratio for this analysis was 5.07 which allows 

rejection of the one-population hypothesis at the .025 level. 

(2) Scheffe Comparison. The Scheffe comparison was con- 
ducted using an F-value corresponding to a significance level of .025. 
This level falls between "statistically significant" and "statistically 
very Significant". Pooled years 1949 - 1951 were found to be signifi- 
cantly different from pooled years 1961 ~ 1963. This finding agrees 
with the significant positive trend identified in the Grumman data in 
paragraph IV.2.a. 

c. Douglas 

Douglas started and completed 2h contracts during the period 
under investigation. These data were pooled in two-year treatments 
and a one-way analysis of variance was performed. The sample sizes, 


means, and standard deviations from this analysis were: 
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Sample Standard 


Treatment _Size_ Mean Deviation 
uo & 1950 3 aly 2 hh 
1951 & 1952 8 ~3.1) ec 25 
os & 195) 6 -3.20 10.00 
1955 & 1956 2 3.45 ea 
1957 & 1958 2 ~8 .00 9.05 
1959 & 1960 2 -1.80 nee 


The F-ratio for this analysis was 0.29 which is not statis- 


t 
tically significant. No Scheffe comparison was conducted in this case. 
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V. CONCLUSIONS 


The most that can be gained from the results of the analyses pre- 
sented in Section IV alone are statistical inferences. Firstly, no 
cause/effect relationships can be established by statistical analysis. 
Secondly, failure to reject a one-population hypothesis or a be = 0 
hypothesis does not prove that those hypotheses were true. It merely 
states that with the statistical methods and standards employed and 
the data analyzed, it was not possible to reject those hypotheses. 
Nevertheless, the author will draw conclusions on the basis that the 
statistical analysis combined with the author's intuition and knowledge, 
and other sources of information present a reasonable case, lacking 
opposing evidence, for the conclusion drawn. The reader who is wunwill- 
ing to accept the author's concineiene on such a basis is free to draw 
his own or none from the statistical evidence presented in Section IV 


of this paper. 


A. CONTRACT PARAMETERS! EFFECT ON DEVIATION FROM TARGET COST 
1. Parameters Which Have a Significant Effect on Contract Cost 
Outcome 
a. Al contractors 
(1) Length of the period of contract performance - Devia- 

tion from target cost varied positively with the length of performance 
of the contract at the rate of approximtely 1.5% per year. The 
author's hypothesis about this variable was supported by the data. 


In view of the length of time available for changes in both requirements 
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and details in a multi-year contract, and the general inflationary 
trend in this country since World War II, this is not a surprising 
conclusion. 

(2) Target profit - Deviation from target profit varied 
negatively as target profit at a rate of more than 1% per percent 
target profit increase. The conclusion that may be drawn here is: 
depending upon the sharing ratio, it may be less expensive in terms 
of total price for the government to be relatively generous in target 
profit. For example, in a contract with a 10% target profit and a 
sharing ratio, a one percent increase in target profit may be expected 
Meeresult in a 1.3% decrease in realized cost and a 0.3% decrease in 
actual price. As the variations in target profit in the data were 
small, one would not be justified in making large target profit adjust- 
ments on this basis. The author's hypothesis with respect to target 
profit was not supported by the statistical analysis. The author has 
no intuitive explanation to offer for this rather surprising result. 

(3) Ceiling price - Deviation from target cost varied 
positively with ceiling price at the rate of approximately 0.h% per 
percent increase in ceiling price. The author's hypothesis with re- 
spect to the effect of ceiling price was supported by the analysis. 

(i) The number of contracts signed in the Navy aircraft 
and missile field during the same year as a particular contract - 
Deviation from target cost varied negatively as the number of contracts 
Signed. This variable was chosen as a measure of the degree of compe- 
tition (or pressure ) felt by a contractor when negotiating a particular 
contract. The theory being that as the total number of contracts in 


the Navy aircraft and missile field becomes smaller, the importance in 
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berms Of prime contractor survival of obtaining a particular contract 
goes up. Reaction to this pressure in the form of acceptance of an 
unrealistically low target cost does not require that the contractor 
knowingly buy in. All he has to do is to relax a little his normal 
degree of resistance to engineering optimism, etc. The author's hypo- 
thesis regarding the effect on deviation from target cost of reduced 
number of available contracts is supported by the analysis. One is 
reminded of the apocryphal story of the gambler who took part in a 
game he knew to be rigged against him because "It was the only game in 
town." The story may be apocryphal but its message is valid. 

(5) The year the contract was signed - The deviation from 
target cost varied positively as the year the contract was signed. 
The author's hypothesis in this regard was supported by the analysis. 
The only explanation that occurs to the author for this result of the 
analysis is a gradual change with time of contractor policy toward cost 
overruns. It is significant that although the year the contract was 
Signed and the number of contracts that year are rather closely related, 
they both are significant in the regression equation. Thus, one may 
not say that the positive effect of the year the contract was signed 
was simply a result of the intensified competitive atmosphere. 

(6) The product of the target cost and the number of 
items procured - Deviation from target cost varied negatively as the 
product of target cost and the number of items procured. The author's 
statistical reasoning concerning a combination of two unrelated vari- 
ables which singly were insignificant was supported by the appearance 
in the regression equation of this variable. The author was then faced 


with the necessity of attempting an explanation of this statistical 
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fact in logical terms. This, he has been unable to do. Nevertheless, 
the predictive power of this variable remains whether or not the author 
can state a logical explanation. 
b. Individual Contractors 

(1) Grumman. Deviation from target cost in the case of 
Grumman contracts showed a strong reaction to both the year the contract 
was Signed and the number of like contracts signed that year. The year 
the contract was signed alone accounted for 0% of the variation in 
Grumman's deviation from target cost. When all of the original variables 
and the two transgenerated ones were considered in a stepwise linear 
regression on the Grumman data, deviation from target cost varied nega- 
tively with the number of contracts signed and positively with the unit 
cost of the item procured (target cost/number of items procured). To- 
gether, these two variables accounted for 57% of the variation in 
Grumman's deviation from target cost. These two variables have approxi- 
mately twice the predictive power in Grumman's case than the variables 
included in the regression equation for all contractors. No conclusions 


were drawn from the second regression (routine forced to use X. per 


9 
IV.A.2.a) because one of the variables included by the stepwise linear 
regression routine had only a .75 probability that its regression 
coefficient (b,) was not equal to zero. 

Grumman's heavy dependence on Navy airframe contracts 
goes a long way toward explaining this result. The complexity involved 
in high unit cost items also provides a reasonable explanation of the 
positive variation of deviation from target cost with unit cost. The 


implications of this predictive equation with respect to the current 


cost overrun problems associated with the F-1]; program are obvious if 
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one allows a valid comparison between non-total package procurement 
contracts and total package procurement ones. 

(2) Lockheed. The impression that Lockheed's deviations 
from target cost were cyclical with time was supported by the linear 
regression which used a cosine function of Xo as the independent 


variable. As compared to a linear regression using X, as the indepen- 


? 
dent variable, the correlation coefficient, determination coefficient 
and F-ratio were materially improved. The extent that the cosine 
equation accounts for the variation, 15%, Lockheed deviation from 
target cost oscillated about a level of -2.3% with an amplitude of 
2.4%. Since the cyclical pattern does not occur when deviation from 
target cost is plotted against the number of contracts signed that 
year, one is tempted to postulate a cyclically varying management 
policy toward cost estimation or negotiated target cost. This theory 
would involve a more optimistic view of probable costs that is tempered 
as the results of the optimism begin to be apparent and then a shift 
back toward optimism as the results of relative pessimism begin to 
surface. 

(3) Lockheed and Grumman. Although the analysis of vari- 
ance conducted using the contractors as treatment levels provided no 
statistical basis for significant difference among contractors, the 
Lockheed and Grumman patterns of deviation from target cost were marked- 
ly different from the other contractors when plotted against the year 
the contract was signed. In both, trends over time were apparent. 

These time trends should be considered when negotiating contracts with 
two contractors. That is, Grumman's tendency to underestimate or accept 


unrealistically low target costs had steadily increased while Lockheed 
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oscillated about a rather favorable level of deviation from target 
cost (-2.3%). 
2. Statistically Insignificant Parameters 

Parameters which are unexpectedly insignificant can be as 
informative as those that prove to be significant. Accordingly, they 
Will be given as much attention as were the significant parameters. 

a. Sharing ratio - The theory of cost-incentive contracting 
rests upon the supposed influence a share of underrun or overrun will 
have upon a contractor's cost-associated decisions. Further, multiple 
incentive contracts depend upon an assumption that the various per- 
formance and schedule incentives will be balanced against the cost 
incentive in the contractor's tradeoff decisions to the advantage of 
the contractor and the government. If the cost incentive proves to be 
ineffective and the results relatively unpredictable, then the supposed 
benefits of multiple incentive contracting must also be seriously 
doubted. This is a point of considerable importance since the addi- 
tional cost of administering a multiple-incentive contract may be 
largely wasted. 

In the case of the 177 Navy aircraft and missile contract 
examined in this thesis, sharing ratio is an insignificant parameter 
with respect to association with cost overrun/underrun (probability 
the Dy paOmeetween 0.50 and 0-75). This 1s nota wholly surprising 
outcome in that a Rand study of 27 Air Force contracts resulted in a 
conclusion that the value of the sharing ratio was an insignificant 
parameter Yeeet. 5 / . Because the Air Force contract data contained 
both cost incentive and cost reimbursable contracts, Rand was also 


able to conclude that although the value of the sharing ratio was 
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insignificant, deviations from target cost were significantly more 
positive in cost-reimbursable contracts than in cost-incentive con- 
tracts. An explanation which might be made is that the existence of 
the incentive completely unrelated to the value of the sharing ratio 
might influence contractor performance of the contract. A more likely 
explanation is that ne US Ceol sanmineenbawe scontractuan 2 particular 
case may have resulted in an upward shift in the negotiated target 
cost Janets. w/ - Since cost-reimbursable contracts encourage contrac- 
tors to grossly understate expected costs to ensure receiving the 
contract, cost-incentive contracts may have the benefit to the govern- 
ment of preventing grossly understated expected costs. 

In any case, the analysis contained herein in no way supports 
the theory that sharing the cost risk with the contractor significantly 
effects his efficiency of performance of the contract or influences him 
in cost-~-associated decision making. 

b. Target cost/Unit target cost - Neither target cost nor unit 
target cost were significantly associated with deviation from target 
cost. Some attractively logical hypotheses regarding these parameters, 
e.g., that lower unit cost should be associated with more negative 
deviation from target cost because of economies of scale and the learn- 
ing curve. That is, one would suppose that the expected positive de- 
viation from target cost would be higher in a $100 M contract for 100 
items (unit cost of $1 M) than a $100 M contract for 10,000 items (unit 
cost of $10,000). That attractive theory must be allowed to wither for 
lack of empirical support. However, one might consider the possibility 
that contractors depend too heavily on economies of scale, etc., in 
bmevr cost estimates, or that the discontinuities in production fre- 


quently resulting from the annual nature of defense funding may tend to 
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prevent gaining the aon advantages of economies of scale or the 
learning curve. 

c. Number of items - Similar arguments to those applied to 
unit cost could be applied to the number of items, i-e., the larger 
the number of items the more negative the deviation from target cost, 
but such arguments were similarly unsupported by the data. 

d. Contractor - No statistical significance could be attached 
to the contrasts among contractors. However, if faced with a choice 
between Kaman (mean percent deviation from target cost = 11.95, standard 
deviation = 12.97) and Vertol (mean percent deviation from target cost = 
-).77, standard deviation = 5.48) other things being essentially equal, 
one would be foolish to choose Kaman on the basis that the difference 
in contract experience with the two contractors was not statistically 
Significant. 

e. Type of item - Although the items procured under the 177 
contracts in the data differ radically in physical terms, they were all 
procured by the same agency, NAVAIRSYSCOM, to aviation standards. No 
Statistically significant differences were found among them. The means 
differ noticeably but ‘the associated variances are too large to allow 


statistically significant differences among the means. 


B. OVERRUN/UNDERRUN PREDICTIVE MODEL (ALL DATA) 


The following discussion refers to the predictive model reported in 


paragraph IV.A.1.a, 
Percent deviation from target cost = 


= 0.20 ee Oreo 0.0000LX, , 


2. + 0.12EX, 2 1.08%, + 0.397X , 9 


D 
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1. Amount of Variation Explained 
The power of the model to explain the variation in the analyzed 
data, approximately 25%, was disappointingly low. Its power to predict 
future contract outcomes, of course, is unknown but should not be ex- 
pected to improve over that for the data from which it was generated. 
Still, a model of this nature might give some inkling of the outcome 
of a particular future contract. It is probably better than an assump- 
tion that the contract outcome in terms of actual cost will be equal to 
the negotiated target cost. The argument here, albeit a weak one, is 
that the model is probably better than no consideration of the histori- 
cal realities of contract outcome. But, of course, past realities may 
or may not be similar to future realities. As is the case with many 
mathematical models, most of the value of this model is in understand- 
ing the relationships or lack of them among the variables. 
2. otandard Error of Estimate 
The large standard error of estimte, approximately 8%, is in 
keeping with the low amount of variation accounted for. The model as 


a valid predictor of future contract outcome is largely useless. 


C. THE RELATIONSHIP OF TARGET PROFIT TO OTHER CONTRACT PARAMETERS 
Three other contract parameters were found to be statistically 
related to negotiated target profit. They were sharing ratio, ceiling 
price, and number of items procured. Together these variables follow 

the previous trend, that they account for little of the variation of 

target profit. This discussion will concern the individual variables. 
1. Sharing ratio - Target profit varied positively with sharing 

ratio. The theory that the contractor who bears the larger financial 


risk should be rewarded with a higher target profit appears to be 
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supported in this case. A similar conclusion was reached by Rand in 
its study of 27 Air Force contracts / ref. 7/. 

2. Ceiling price - The negotiated target profit varied negatively 
with ceiling price. A higher ceiling price reduces the contractor's 
financial risk by causing the point of total assumption (PTA) to occur 
further from the target cost. All other things being equal, the further 
the PTA is from the target cost, the lower the probability of the con- 
tractor finding himself in the position of paying excess costs out of 
his own pocket, dollar for dollar. Again, the lower the financial risk, 
the lower target profit, and vice versa. 

3. Number of items procured - The target profit varied positively 
as the number of items procured. No explanation for this phenomena was 


apparent to the author. 


D. SUMMARY OF CONCLUSIONS 

The statistical inference really worthy of note in this thesis is 
the repeated failure of empirical evidence to support the supposed 
effectiveness of the sharing ratio. Since the cost incentive is the 
cornerstone of incentive contracting and incentive contracting is in 
extensive use today, it is a little disconcerting to find no empiricas 
evidence to support it, 1.e., percent deviation from target cost is 
not statistically related to the sharing ratio. It would seem prudent, 
lacking supportive evidence, to not place such dependence on incentive 
contracting to influence efficient contractor performance. Faith in 
an unprovable principle has its place in religious matters but not, in 


the author's view, in matters of economy and national defense. 
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APPENDIX A 


t 
COMPUTER CODE FOR THE SCHEFFE 


COMPARISON (S-TEST) 


The following computer code is written in FORTRAN G and was used on 


an IBM 360 computer. 


@ PROGRAM TO TEST FOR SIGNIFICANT DIFFERENT IN ANOVA TREATMENTS 
c USING SCHEFFE METHOD 
READ (5,10) NGROUP,SQUARE, FVAL 
10 FORMAT (12,2F10.)) 
DIMENSION ISAMSZ (30), ERMEAN(30),ERTOT(30) 
DO 15 J=1,NGROUP 
READ (5,12) ISAMZ(J),ERMEAN(J) 
12 FORMAT (I2,F12.5) 
ERTOT(J )=ISAMSZ (J )+ERMEAN (J ) 
15 CONTINUE 
JGROUP=NGROUP-1 
A=(JGROUP#FVAL )2+0.5 
DO 200 I=1,JGROUP 
DO 150 J=I,JGROUP 
K=J +1 
COMP=ISAMSZ (K )*ERTOT(I )-TSAMSZ (I )*ERTOT(K ) 


IF (COMP.GE.O) GO TO 152 
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COMP=-COMP 

152 VARCOM=ISAMSZ (K )+*2*ISAMSZ (I )#SQUARE+ISAMSZ (I ):+:2*1T SAMSZ(K )+SQUARE 
AVAR=A*VARCOM+*0.5 
WRITE (6,110) I,K,COMP,AVAR 

110 FORMAT ('O', 'GROUP!',2X,12,)X,'GROUP! ,2X,12,))X, 'COMPARISON=', 
1F12.),)X, 'AVARTANCE=! ,F12.),) 
IF (COMP.LE.AVAR) GO TO 150 
WRITE (6,120) I,K 

120 FORMAT ('0!, !ss4%* GROUP',1X,12,1X,'IS SIGNIFICANTLY DIFFERENT FROM 


1 GROUP'1X,12,!' xe!) 


150 CONTINUE 
200 CONTINUE 

STOP 

END 

Explanation of Input Variables 
NGROUP Number of treatment groups (levels) 
SQUARE Within-groups mean square value from 
preceeding ANOVA 

FVAL I sezhivn fo atishe aesainea level of significance 


of the comparison based on between-groups 
and within-groups degrees of freedom from 
the ANOVA. Example: If the between groups 
degrees of freedom are 3 and the within- 
groups 18, and the desired level of test is 


.025, the value of FVAL is 3.9539. 
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(F5 4g; o9¢7 3+9539)- 


TSAMSZ (J) The sample size in the jth treatment group 
ERMEAN (J ) The mean of the jth treatment group 
Limitations 


The program as contained herein is limited to: (1) no more than 
30 treatment groups and accompanying values of ISAMSZ and ERMEAN, and 
(2) comparison of each treatment group with each of the other treat- 


ment groups. 


Sample Output 


GROUP 1 GROUP COMPARISON= 93.3989 AVARIANCE= 36.9731 
xxx* GROUP 1 IS SIGNIFICANTLY DIFFERENT FROM GROUP h +HHte 


GROUP Q GROUP 3 eseeeeeoeo see eee enee ee ae ee eoev,eeveveevesrvr, ee ec eenvneeeees 


5h 





LIST OF REFERENCES 


Scherer, F.M., "The Theory of Contractual Incentives for Cost 


Reduction," The Quarterly Journal of Economics, v. 78, p. 267, 
May 196h. 


ocherer, F.M., "The Theory of Contractual Incentives for Cost 


Reduction," The Quarterly Journal of Economics, v. 78, p. 262, 
May 196). 


Rand Corporation Report RM-5120-PR, Cost Incentives and Contract 
Outcomes: an Empirical Analysis, by Irving N. Fisher, p. 1], 
September 1966. 


McCall, J.J., "The Simple Economics of Incentive Contracting," 
The American Economic Review, v. 60, number 5, p. 8l5, 
December 1970. 


Rand Corporation Report RM-5120-PR, Cost Incentives and Contract 
Outcomes: an Empirical Analysis, by Irving N. Fisher, p. 35, 
September 1966. 


Rand Corporation Report RM-5120-PR, Cost Incentives and Contract 
Outcomes: an Empirical Analysis, by Irving N. Fisher, p. 13, 


— 


September 19606. 





Rand Corporation Report RM-5120-PR, Cost Incentives and Contract 
Outcomes: an Empirical Analysis, by Irving N. Fisher, p. 26, 
September 1966. 


\ 


a 





INITIAL DISTRIBUTION LIST 


No. 


Defense Documentation Center 
Cameron Station 
Alexandria, Virginia 2231) 


Library, Code 0212 
Naval Postgraduate School 
Monterey, California 9390 


Associate Professor Michael G. Sovereign 
Code 55Zo 

Department of OR/AS 

Naval Postgraduate School 

Monterey, California 9390 


Professor P.A.W. Lewis 

Code 5S5SLw 

Department of OR/AS 

Naval Postgraduate School 
Monterey, California 93910 


Mew Moh. Bireroecehi 

Code MAT 022B 

Headquarters, Naval Material Command 
Washington, D.C. 20360 


Assistant Professor D.R. Whipple 
Code 55Wp 

Department of OR/AS 

Naval Postgraduate School 
Monterey, California 9390 


Department of OR/AS 


Naval Postgraduate School 
Monterey, California 9390 


56 


Copies 










DOCUMENT CONTROL DATA-R&D 
tSecurity classification of title, body of ebstract and indexing annotation must be entered when the overall report is classified) 
a Po maw 
; a e $4 
Yzal Postgraduate School Unclassified 


MEORT TITLE 


tatistical Analysis of Deviations from Target Cost in NAVAIRSYSCOMHQ Fixed-Price 
Esentive Contracts During the 1949 - 1965 Time Frame 


Security Classification 


Ca 












CRIPTIVE NOTES (Type of report end inclusive dates) 


Ster's Thesis; March 1°73 
FHORIS) (First name, middie initial, lest name) 


x Wayne Dixon 


4 
2ORT DATE 7a. TOTAL NO. OF PAGES 7b. NO. OF REFS 
gen 1973 | 58 t | 


IN TRACT OR GRANT NO. 9@, ORIGINATOR'S REPORT NUMBER(3) 





ROJECT NO. 


9b. OTHER REPORT NOUS) (Any other numbers that may be eeelgned 
thia report) 


ISTRIBUTION STATEMENT 


Approved for public release; distribution unlimited. 


UPPLEMENTARY NOTES 12. SPONSORING MILITARY ACTIVITY 


Naval Postgraduate School 
Monterey, California 9390 





BSSTRACT 


This paper statistically analyzes 15 years of NAVAIRSYSCOMHQ fixed-price incen- 
ive contract experience in the aircraft and missile procurement field. The relation 
f basic contract parameters to contract outcome is explored through regression and 
1alysis of variance techniques. 


The inferences arising from the statistical analysis are combined with other 
aformation to draw conclusions regarding incentive contracting. The most important 
f these is that there is no evidence that the negotiated sharing ratio has any 
afluence on the contractor during the performance of the contract. 


ae 


) 1nNOov wiGl3 (aAGE hy 07 
0101-807-6811 Security Classification 


A-31408 








Security Classification 


KEY WORDS 


Statistical Analysis 
Cost-Incentive Contracting 
Fixed-Price Incentive 

Sharing Ratio 

Aircraft and Missile Contracts 
Naval Air Systems Command 
Seiling Price 

Target Profit 

Grumman 


Lockheed 


ror’ 1473 (eack) 


O0101+807-6821 


58 





Security Classification 


A-31409 

















Tests 5308 


D599 Dixon 

e.1 A statistical analysis 
of deviations from target 
cost in NAVAIRSYSCOMHO 
fixed-price incentive 
contracts during the 
1949-1965 time frame. 


DUDLEY KNOX LIBRARY 


ae a aa hia 





